Papers with SOTA 7B models
Evolutionary Contrastive Distillation for Language Model Alignment (2024.findings-emnlp)
Copied to clipboard
Julian Katz-Samuels, Zheng Li, Hyokun Yun, Priyanka Nigam, Yi Xu, Vaclav Petricek, Bing Yin, Trishul Chilimbi
| Challenge: | Existing studies indicate that large language models struggle with challenging instructions. |
| Approach: | They propose a method for generating high-quality synthetic preference data to enhance the complex instruction-following capability of language models. |
| Outcome: | The proposed method exceeds the performance of current SOTA 7B models and is competitive even with open-source 70B models. |
LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing LMM-based embedding models exhibit a high degree of overlap in similarity distribution between positive and negative pairs, making it challenging to distinguish hard negative pairs effectively. |
| Approach: | They propose a framework that improves the embedding model's representation learning for negative pairs based on their discriminative difficulty. |
| Outcome: | The proposed framework improves the embedding model's representation learning for negative pairs based on their discriminative difficulty. |